Network abnormal traffic detection is an important part of network security protection. At present， abnormal traffic detection methods based on deep learning treat the port number attribute the same as other traffic attributes， ignoring the importance of the port number. Considering the idea of attention， a novel abnormal traffic detection module based on Convolutional Neural Network （CNN） combining Port Attention Module （PAM） and Convolutional Block Attention Module （CBAM） was proposed to improve the performance of abnormal traffic detection. Firstly， the original network traffic was taken as the input of PAM， the port number attribute was separated and sent to the full connected layer， and the learned port attention weight value was obtained， and the traffic data after port attention was output by dot-multiplying with other traffic attributes. Then， the traffic data was converted into a grayscale map， and CNN and CBAM were used to extract the the channel and space information of the feature map more fully. Finally， the focus loss function was used to solve the problem of data imbalance. The proposed PAM has the advantages of few parameters， plug and play， and universal applicability. The accuracy of the proposed model is 99.18% for the binary-class classification task of abnormal traffic detection and 99.07% for the multi-class classification task on the CICIDS2017 dataset， and it also has a high recognition rate for classes with only a few training samples.